For investment bankers, a Brexit vote is more than just about volatile prices on a screen but also about where they may end up working and living.